feat: Keck NIRC2 AO reduction (phase 4) — KOA acquire, ground stages, nirc2_native combine, provisional PSF - #12
Merged
Conversation
… nirc2_native combine, provisional PSF Implements K1-K3 of the design on #9 (per #11), on the post-phase-3 layout: - acquire-backend seam (archive: mast|koa); acquire/koa.py fetches raw level-0 science + ITIME/COADDS-matched darks + nearest-night flats (+/-14 d) + PSF-star frames + epoch-matched distortion solutions (Yelda 2010 / Service 2016, checksummed), self-healing on re-runs - ground stages: calibrate/ (DN->e-, dark, lamp-on-off flat, bad pixels) and sky/ (scaled running sky, 2 passes — drift-unbiased) - nirc2_native combine: distortion + phase-correlation registration + rescale as one drizzle pixmap; IVM weights make the shared noise recipe apply verbatim (same Casertano R) - sampling-mode-aware read noise (MCDS/Fowler: RN_cds/sqrt(MULTISAM)); per-frame budget validated on SHARP B1938 K' (62.0 vs 62-64 e-) - provisional-PSF contract: tier-A PSF-star epochs, coherence + diffraction-floor vetting (rejects cosmic-ray epochs), every surviving epoch shipped as psf_candidate_<i>.fits, psf_provisional provenance - docs/design/keck_ao.md (closes the #9 write leg) + roadmap + AGENTS.md - validated end-to-end on the SHARP I B1938+666 discovery data (U085N2L): closure 0.841, PSF FWHM 72 mas, WHT uniformity 0.084, 39x180 s Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ovenance Review faculty pass (8 angles, 30 candidates -> 10 verified findings, 9 fixed): - sky windows now split at time gaps (nights / PSF-star visits); single-frame recipe moved into the sky module; shared mad_sigma helper - keck tier-B ePSF regains the per-exposure saturation cut - KOA acquire: pointing-coherence guard (the footprint-filter analogue), distortion-epoch boundary guard, lower(koaimtyp) normalisation - nirc2_combine: per-frame distortion consistency check, distortion-derived grid margin, hoisted mgrid; detector constants now adapter-routed (roadmap boundary rule: nothing outside instruments/ names a detector) - provenance: absolute distortion paths dropped from reduction.json; stale psf_candidate files cleared on any keck re-run; candidates in products - re-validated on B1938+666: closure 0.883, PSF FWHM 72 mas, 110 tests pass Deferred to keck_ao.md open items: orchestrator dispatch generalisation, prepared-header schema, header-GAIN cross-check, FWHM unification. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: Keck NIRC2 AO reduction (phase 4) — KOA acquire, ground stages, nirc2_native combine, provisional PSF
Implements phases K1–K3 of the design on #9 (research task) per #11, on the post-phase-3 consolidated layout (#10). SHARP-grounded (Lagattuta et al. 2012; Chen et al. 2016, 2019); decisions adopted from #9's batched set: drizzle-package combine backend, B1938+666 validation anchor.
What's new
InstrumentAdapter.archive(mast|koa);acquire/koa.py(PyKOA TAP) fetches raw level-0 science frames, the night's darks (ITIME/COADDS-matched) + flats (nearest flat-bearing night within ±14 days when the science night has none — routine for NIR), PSF-star frames, and the epoch-matched NIRC2 distortion solution (Yelda 2010 / Service 2016 lookup tables from the canonical jluastro/nirc2_distortion distribution, checksummed into provenance). Components self-heal independently on cached re-runs.calibrate/(DN→e⁻, optional matched dark, lamp-on−lamp-off flat, hot/dead bad-pixel mask → NaN → zero drizzle weight) andsky/(scaled running sky: unit-median structure from object-masked temporally adjacent frames × the frame's own masked median, two passes — kills the edge-of-sequence bias of a plain running median under a drifting K′ sky).nirc2_nativecombine backend: distortion + phase-correlation registration + native→final rescale as one drizzle pixmap through the standalonedrizzlepackage (same engine as drizzlepac/jwst-resample), per-frame weights = inverse background variance → the weight map is the IVM the unchanged shared noise recipe (noise.rms.noise_map_from, same Casertano R) expects. 10 mas narrow / 40 mas wide final scales (SHARP convention).psf_candidate_<i>.fits(final selection belongs to modelling — SHARP I's evidence-based practice);psf.fitscut from the sharpest epoch;psf_provisional: truein provenance. Tier B (in-field ePSF) also flagged provisional on keck.docs/design/keck_ao.md(per-stage deltas, detector constants, open items: wide camera, subarrays, MULTISAM read noise, north-up resampling) — closes the parked write leg of research: plan the Keck NIRC2 AO reduction workflow (SHARP context) #9. Roadmap + AGENTS.md updated.keck = ["pykoa>=1.7", "drizzle>=2.0"]extra. AGENTS.md test-boundary note: the lightweightdrizzleresampler is sanctioned in unit tests behindimportorskip(drizzlepac/jwst stack still excluded).Validation — B1938+666 (SHARP I discovery data)
prototypes/b1938_keck_spike.py: KOA discovery found the actual SHARP I dataset (program U085N2L, UT 2010-06-29/30; 39×180s lens frames; PSF/tip-tilt star pointings at 20″ interleaved in time, exactly as published). Acceptance checks (keck_ao.md):Acceptance items 3-4 (astrometric parity vs HST; end-to-end PyAutoLens Einstein-radius invariance) follow as their own prompt once this dataset enters a modelling run.
Tests
110 pass in the task worktree (24 new Keck tests). Downstream suites n/a — nothing imports
autoreduce. Smoke n/a — no workspace script surface.Unit tests are numpy/astropy(+drizzle-behind-importorskip) synthetic-frame tests: calibrate math, scaled-sky recovery under drift, sub-pixel registration convention, flux conservation + EXPTIME through the pixmap, noise closure with the read-noise-dominated NIRC2 budget, epoch grouping + candidate selection, loud-failure paths (wide camera, mixed setups, garbage flats).
Review + gate
Review faculty: 8 angles -> 30 candidates -> 10 verified findings, 9 fixed in the second commit (epoch-aware sky grouping, tier-B saturation cut, pointing-coherence guard, portable provenance, distortion-epoch guards, adapter detector boundary, stale-candidate hygiene); 1 deferred as documented keck_ao.md open items. Post-fix verdict CLEAN. Heart YELLOW (6 chronic organism reasons, none touching this repo) acknowledged by the maintainer at sign-off ("ship + merge", 2026-07-09).
--autosupervised run; human-directed ship and merge.🤖 Generated with Claude Code